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ABSTRACT 


A detailed analysis of the performance the Wiener optimal filter for estimating a 
signal in additive noise is carried out. A first order AR model is assumed for both the 
signal and noise. Both ITR and FIR forms of the filter are considered and expressions are 
derived for the processing gain, mean-square error and signal distortion. These measures 
are plotted as a function of the model parameters. 

This analysis motivates a generalized form of the Wiener filter, which can improve 
the signal distortion. An analysis of this more general filter is then carried out. 

A practical noise removal algorithm based on short-time filtering using the gen- 
eralized filter is also described, and results of applying the algorithm to some typical 


underwater acoustic data are presented. 
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I. INTRODUCTION 


A. MOTIVATION FOR THE STUDY 


Data from passive sonar is generally accompanied by ambient noise arising from 
shipping traffic, marine life, wave motion, moving or cracking ice (in the Arctic and 
Antarctic), and numerous other sources. The statistical properties of the noise are vari- 
able, even direction-dependent, and have been the source of many studies and analyses 
[Refs. 1, 2, and 3]. Noise degrades sonar data collection and related processing of the 
data to extract information. 

Since ambient noise cannot be completely avoided when collecting real underwa- 
ter acoustic data, it is desired, in many situations, to remove the noise before further 
processing. In general, the signals of interest and the ambient noise are non-stationary 
and their statistics are not known a priori. Here we approach the problem of removing 
additive noise from a given signal by using a short-time optimal filtering technique. In 
this approach we assume that the noise is stationary for the duration of the signal, and 
that the signal can be assumed stationary over very short time intervals. We exploit this 
feature and develop improved algorithms for removing the noise. 

The work in this thesis basically consists of two parts. In the first part (Chapters 
II and III), we perform a detailed analysis of the Wiener filter and evaluate its perfor- 
mance using three different criteria. For this work we use a simple signal model which 
nevertheless provides insight into results for more general types of signals. Both the IIR 
and FIR forms of the filter are considered. In the second part (Chapters IV and V), we 
propose a new algorithm based on a generalization of the Wiener filter and motivated by 
our analysis, that can be applied to real data. The implementation of the algorithm on a 
short-time basis is similar to that developed by Frack [Ref. 4] and we use the same basic 
structure of the programs. Results of the application of this algorithm to underwater 


acoustic data (biologic data) are presented. 


B. PROBLEM DESCRIPTION 
In this section we describe the problem of noise removal to be addressed and 
introduce the Infinite Impulse Response (IIR) and the Finite Impulse Response (FIR) 


forms of the Wiener filter used to estimate the signal in noise. 


1. Problem Statement 
We consider here the problem of estimating a signal in additive noise. The ob- 


served discrete observation sequence is given by 


x(n) = s(n) + n(n) (1.1) 


where s(n) is the signal and 7(n) is the noise. The signal duration may be anywhere 
from a few miliseconds to a few seconds and is generally a none rOney random process. 
The noise is assumed to be stationary over the entire observation interval and may be 
observed without the signal during the early part of the observation interval. That is, 
the signal becomes non-zero some time after the beginning of the observation interval, 
and the precise time at which it becomes non-zero is not known. Statistics for the signal 
and noise, such as mean or correlation function, are not known a priori and must be 
estimated from the data. 

Let us assume that the signal is to be estimated by applying the observation 
sequence to a linear filter as shown in Figure 1, and the filter is designed to minimize the 


mean-square error 
? = E4(s(n) — 8(n))?} 
where §(n) = y(n) is the filter output. Then the solution for this problem is obtained by 


solving the Wiener-Hopf equation [Ref. 5], which for the case of a stationary signal has 


the form 


4 


R,(l — i)All] = Ryx(i); O0<i<co (1.2) 


mney 
Il 
°o 


Here R,(l) is the correlation function of the observations, h(l) is the impulse response 


of the filter, and R,,(l) is the cross-correlation function between the signal and the 


x(n) = s(n) + n(n) y|n] = 8(n) 


Figure 1. General Linear Signal Estimation Problem. 


observation sequence. Since the noise is assumed independent of the signal, it is easy to 


show that 
R,(l) = R,(l) + R, (0) (1.3) 
and 


Tea) = Rh, (1) (1.4) 


where FR, (1) and R,(l) are the signal and noise correlation functions, respectively. The 


estimate for the signal is then given by 
a(n) = y(n) = > h(n—k)a(k) (1.5) 
k=—0o 


and the mean-square error can be shown to be 
o? = R,(0) — S- A(1) Rea (2) (1.6) 


where again R,,(l) = R,(l) for this problem. 

When the signal s(n) is nonstationary, as is generally the case in our application, 
the optimal filter is time-varying and equations (1.2) through (1.6) take on a slightly 
different form [Ref. 5] involving the time varying correlation functions R,(n,,n2) and 
R,(n 1,2). Although these correlation functions could be estimated from the data and 
the corresponding time-varying filter could be computed, it is generally more efficient to 
take a block-oriented approach where the data is assumed stationary over short segments 
and the equations for a time-invariant filter are used to find a filter that applies only to 
the data segment. Although we do not do so in this work, the data segments could be 


overlapped by some amount, and indeed when the overlap consists of all but the a single 


point, the result is equivalent to using the time-varying filter. Thus for purposes of our 
work and analysis here, (1.2) through (1.6) are the relevant equations. 

The Wiener-Hopf equation (1.5) can be solved for two types of filters, the Infinite 
Impulse Response (IIR) filter, which is applied recursively to the data, and the Finite 
Impulse Response (FIR) filter, where the filtering process involves a simple convolution. 
Both of these filters are known as Wiener optimal filters and are described in following 
sections. Later, we investigate the performance of these two filters in the case when both 
signal and noise are represented by a first order autoregressive (AR) model. This inves- 
tigation using a simple model provides insight into the general behavior of the Wiener 


Filter, and so, helps to predict its behavior for more involved cases. 


2. Solution of The Wiener-Hopf Equations: 
The IIR Filter 


The IIR Wiener filter is recursive in form and can have the advantage of requiring 
fewer parameters than a comparable FIR form of the filter. In many cases the IIR filter 
is the only true optimal solution to the problem and FIR filters are in fact suboptimal 
since they do not achieve the absolutely lowest mean-square error. Since the filter is 
required to be causal, the problem is not straightforward; it was solved (originally in the 
continuous case) by Wiener using spectral factorization methods [Ref. 6]. Because the 
solution procedure will be referred to later in this work, we summarize it in this section. 

The solution for the IIR filter derives from observing that if the observation se- 
quence is a white noise process R,(l) = 025(1) then (I.2) becomes 

5 o26(I =i) =e) 0 < 1 aieo (Ee?) 
1=0 
leading to the simple solution 
a(t) = aRea(!) 120 (1.8) 
0 L<0 
The procedure then is to first whiten the observed random process x and then apply the 


solution above. Thus we conceptually represent the optimal filter h as a cascade of two 


filters, the whitening filter g, and the optimal filter h for the whitened process. Both g 
and hare required to be causal. 
To find the whitening process we observe [Ref. 5] that the complex spectral density 


function of the input process x can be factored as 
SF Zamna caleg lnzlZeaee (1.9) 


Therefore, if we choose G(z) = 1/H,,(z), we see that the filter g will have the necessary 
whitening properties and the whitened input process will have variance Ko. 

The cross-correlation between the whitened input process and the signal has a 
complex cross-spectral density function given by S,2(z)/Hea(z~'); therefore, the filter 
corresponding to (1.8) for the whitened process is written as 

1 ee 
H (z) = K ae (I.10) 
where the notation [| |, means that the resulting function corresponds only to the causal 
part of the quantity within brackets. Finally, by cascading G(z) and H (z) we obtain the 
expression for the optimal IIR filter 


H(z) = a ae. (I.11) 


where H,g(z) and Ko are derived from the spectral factorization (I.9), and S,,(z) is the 


cross spectral density function for s(n) and z(n). 


be Solution of The Wiener-Hopf Equations: 
The FIR Filter 


The FIR filter is simpler to derive. Since the filter has finite length P, the filtering 


equation (1.5) can be written as 


"Gl 3 eee (1.12) 


We can also directly apply (1.2) noting that the upper and lower limits need to reflect 
the causality and finite length of the filter. The Wiener-Hopf equation for real process 


then becomes 


> R,(1—ah(0) =R,.(i); i=0,1,...,P-1 (1.13) 


l=0 


and the mean-square error (1.6) is given by 


a? = R,(0) ~ ~ A() Rec(l) (1.14) 


For the FIR filter it is most convenient to write the equations using vector nota- 


tion. The filter output is written as 
y[n] = h?x (1.15) 


where h and x are the vectors of filter coefficients and observations respectively 


h(0) x(n) 
— ny) _— x(n _ ie 
mae — 1) t(n— P +1) 


Equations (1.13) and (1.14) can then be written as 
R,h =,, (1.16) 


and 


o? = R,(0) —h’ i, (1.17) 


where R, = E {&(n)&7(n)} and r,, = E{s(n)x(n)}. Since the signal and noise are 
uncorrelated, we have 


R,=R, +R, (1.18) 


where R, and R, are the signal and noise correlation matrices, and rz is the first column 
of R,. This form of the equations is used in the analysis that is presented later in this 


thesis. 


C. ANALYSIS MODELS 


The detailed analyses presented in this thesis are based on a first order autore- 
gressive (AR) model for the signal and the noise. The biological signals and the ocean 
noise tend to have a Gaussian distribution. The AR model is a linear model, so when 
modeling these signals and noise it preserves their Gaussian nature. Also, the solution 
of AR model depends only on a system of linear equations, allowing a minimum phase 
solution, i.e., providing a causal stable model with a causal stable inverse (Ref. 5]. The 


signal s(n) is assumed to be generated according to the difference equation 
s(n) = as(n — 1) + u,(n) (1.19) 


where w,(n) is a (zero mean) white noise process with variance o? 


w, and a is a real valued 


parameter with |a| < 1. The correlation function for the signal is then found to be 
R,(l) = 02a"! (1.20) 


where the signal power ? is given by 


2 
CO 
a (1.21) 


Thus a is seen to represent the correlation coefficient for the process (a = R(1)/R(0)) 
with higher values of a representing increased correlation. The complex spectral density 
function for the signal is the z-transform of the correlation function R,(/) and has the 


form 
_* of (1-7) 
22 eae 


If this is evaluated on the unit circle (z = e7”) we have the power spectral density function 


(1.22) 


_ a2 (1-a?) 
~ 1+a2 — 2acos(w) 


S;(e”) (1.23) 


For a > 0 this spectrum has a lowpass character while for a < 0 the spectrum has a 
highpass character (see Figure 2). For a = 0 the signal is a white noise sequence and the 


spectrum is flat. 


— alpha>0 
-- alpha<0O 


power spectral density function 





-3 -2 —1 0 1 2 3 
frequency (omega) 


Figure 2. Power Spectral Density Function for Real Exponential Correlation Function 
(a >O and a <0). 


The noise n(n), when it is not white, is represented by a similar model. The noise 


satisfies the difference equation 
n(n) = yn(n — 1) + w,(n) (1.24) 


where w,(n) is a white noise process independent of w,(n), and ¥ is real with magnitude 


less than 1. The process has a correlation function of the form 
R(t) = of"! (1.25) 


where the noise power o¢ is given by 





= — (1.26) 


and a complex spectral density function 
Onn =a) 


2) = Gal 1) 


(ea) 





a(n) = s(n) + n(n) y(n) = y.(n) + yn(n) 
 daieniainamaaciiaial (Zz) 


Figure 3. General Linear Signal Estimation Problem. 


The power spectral density function is therefore given by 


oy (1-7) 


Ss 
le) 1+ 7? — 2ycos(w) 


(1.28) 


and has a lowpass or highpass character according to the sign of -y. As described earlier, 
the noise is assumed to be added to the signal, and the main problem addressed in this 


thesis is the removal of the noise or estimation of the signal. 


D. FILTERING AND MEASURES OF PERFORMANCE 

We have considered the problem where we observe the sequence x(n) given in (I.1) 
where s(n) is the desired signal and 7(n) is additive noise uncorrelated with the signal. It 
is desired to remove as much of the noise as possible by linear filtering. Although we have 
already discussed the optimal solution in terms of mean-square error, let us temporarily 
put that solution aside and formulate the problem more generally. We assume that the 
observed sequence z(n) is applied to a filter and the output of the filter y(n) is an estimate 
of the signal s(n). Since the filter is linear, we can define components of the output as 
shown in Figure 3. In particular y,(n) is the response of the filter to the signal alone and 
y,(n) is the response to the noise alone. These components permit defining the following 
measures of performance: 

Processing Gain: The processing gain (PG) for the filter is defined as 

pees (1.29) 
Pin 

where Pin and Poy; are the input and output signal-to-noise ratios defined as 


og, = Fal) _ Ets%(n)} 
= R,(0)  B{n?(n)} 





(1.30) 


ee E Se a 





Pout = (I3h) 
The processing gain is usually measured in dB, i.e., 
PG(dB) = 10 logy, = (1.32) 


Processing gain relates to the relative power in the signal and the noise but does not 
address accuracy of the signal estimate. 

Mean Square Error: The mean-square error is the estimated value of the 
squared difference between the signal and the filtered output. Here we use a normal- 
ized measure of the mean-square error defined by 

use — EL(s(n) = u(n))"} _ Re(0) 

E{s*(n)} R,(0) 

where R,(l) is the correlation function for the “error” defined as e(n) = s(n) — &(n) = 
s(n) — y(n). 

When the output of the filter is equal to the signal itself (except possibly at a 
countable set of points) then the mean-square error is equal to zero. The maximum 
occurs when y(n) is zero, where this normalized mean-square error has a value of 1. 
Mean-square error is the quantity that is optimized in the traditional Wiener optimal 
filtering problem discussed above. 

Signal Distortion: Signal distortion (SD) addresses what the fer does to the 


signal itself irrespective of the noise, and is defined as 


sp a1 Eker) _ Re. 


E{s*(n)}E {y2(n)} (0) Ry, (0) 


where R,,,(/) is the cross-correlation function between s(n) and y,(n). This quantity 


(1.34) 


takes on values in the interval [0,1] and is minimized when the signal is unchanged by 
the filter (y,(n) = s(n)), i-e., there is no distortion introduced. 
An ideal filter would form a perfect estimate for the signal and completely elimi- 


nate the noise. Such a filter would have infinite processing gain, zero mean-square error, 
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and zero signal distortion. Since no linear filter can achieve such ideal performance, 
we can balance these measures, depending on the application, emphasizing one measure 
compared to the others. Thus all of the measures defined above will have use in the 


sequel. 


E. THESIS OUTLINE 


The remainder of this thesis is organized as follows. Chapter II introduces, anal- 
yses and discusses the performance of the IIR Wiener filter described above. Chapter 
III provides a similar analysis and discussion of the performance for the FIR form of 
the Wiener filter. Chapter IV presents a filter based on a new criterion that serves as a 
generalization of the FIR Wiener filter. This filter, which we call the extended optimal 
filter, provides a way to improve the signal distortion with only slight degradation in the 
other performance measures. Chapter V incorporates this technique in a noise removal 
algorithm based on short term filtering of the observation sequence and demonstrates 
its performance on sonar data. Chapter VI concludes the thesis with a summary and 


suggestions for further investigation. 


Jr 





IT. ANALYSIS OF THE ITR WIENER FILTER 


In this chapter we derive analytical expressions for the Wiener IIR optimal filter 
for a signal in colored noise. The signal and the noise are each represented by a first 
order AR model (equations (I.19) and (1.24)). The expressions for the filter generalize 
the results obtained in [Ref. 5] which treats the white noise case only. 

After deriving an expression for the optimal filter we derive the corresponding 
expressions for the measures of performance outlined in Chapter I and examine these 
expressions as a function of some of the parameters describing the model. While the 
model expressed by (1.19) and (1.24) is only of first order, it exhibits many of the gen- 
eral characteristics of a more complicated model and allows us to draw several general 


conclusions. 


A. THE IIR WIENER FILTER 
The general expression for the optimal IIR (Wiener) filter for estimating a real- 
valued signal s(n) in additive noise y|n] (see (I.1)) is given by equation (1.11). For the 
case of a first order AR model, the signal and noise complex spectral density functions 
are given by (1.22) and (1.27) respectively. Since the signal and noise are independent 
with zero mean, it follows from (1.4) that the complex spectral density function between 
the signal s(n) and the observation x(n) is given by | 
2 2 
SS... oe qo en ea) 
Similarly, it follows from (1.3) that the complex spectral density function for the obser- 


vations 1S 


Sz(z) Ss(z) + Sp(z) 
o(1—a?) (1-97) 
(l—az\(l-—az1) (l--yz)(1 — 271) 
Fel lols LE Y= 0 — 7 a) ce 
(1 — az!)(1 — az)(1 — yz71)(1 -— yz) 
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By collecting terms, this last equation can be written as 


—Az+B-— Az} 
5z(z) = oo 
where 
A = yo%(1—a*)+ aos (1 — 7°) 
B= of(1-a*)(1+7’)+o7(1-7)(1 +0’) 
and 


D(z) = (l= az (lazy) —yz~ t= yz) 


After factoring the numerator, we can write (II.2) as 


—Az} 
S.(z) — oT. (z = TNC — Z1p) 
where 
na= 5 (C-VeF—4) 
1 
Ab 5 (C+ VC? — 4) 
and 


Bo, ee lager cn = 7) es 


ca “ou Go. (a) tO ea ae 


This last constant can be expressed as a function of a, yy, and pin, as 


Ca nll = oA A+ Y= 1) 4 (1+ a?) 
(Pin (l= Ga) (Eo lene 





where Pin 1S the input signal-to-noise ratio 
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(11.2) 


(11.3) 


(11.4) 


(II.5) 


(11.6) 


(11.7) 


(11.8) 


(II.9) 


(II.10) 


(11.11) 


From the form of the polynomial in (II.2) and (11.6), it can be seen that 2,2), = 1, or 
that zp = 1/z:,. Let us denote the root that is inside the unit circle in the complex 


plane as 2}, 1.e., 
Ze iW Zinle 1 
a=l 1a (11.12) 
Zp =1/z1¢ otherwise 
Then from (II.10), we see that z,; depends only on a, 7, and pin. 
We can now factor the complex spectral density function S,(z) by writing it in 
the form 
A (1 =2i2") (1 — zz) 


S(2) =F Ter ii-w) G—aal—w — 


In comparing this with (1.9) we can identify 


Ko = A/z (1.14) 
and 
_ (1 — z,z7") 
es) =a (11S) 
Sal Ga) 
Let us now proceed to compute the term oy that appears in (1.11), the 
ca + 


equation for the optimal filter. From (IJ.1) and (JI.15), we have 


Ssa(Z) ee aie }a|?) (1 — az)(1 — yz) 
fiat (1 — az)(1 — az!) (1 — zz) 
= o7(1 _ |a.|?) Ye (1 = (1/y)27*) 


mz (1 —az-!)(1 — (1/z,)z71) 


This can be written in a partial fraction expansion as 


ee _ C} C5 
Hea(z-!) (1 — az-?) A (lel 2) Weer) = 
where 
Ss pepdia 1 /o7) 
C} == o-(1 = la| ) z (1—1/za) (II.17) 
and 
6, = 670 — la (11.18) 


Zz) (1 —_ Za) 
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The causal part of the expression (II.16) can now be written as 


H'(z) = - | el | (II.19) 


C} 


eye 


a 1-lay o2(1— jal?) v/a 


A 1-1/za 1— az! 


zy a—1/y o%(1—- lal’) y/a 


~ Aa 1/z 1— az"! ee) 
Finally, combining (II.15) and (II.20) we obtain 
/ 1 1 ere) 
H(z) = Hea(z) iG, ace 
_  (L eee ari tee Os — [ai") ¥/ 21 
7 (1 — y2z7!) Aa-1/y (1-az-!) 
_ f AO le ees? 9 
= (gaan =a o<(1 — |al*) v/s) (; = ee) (e244) 


The final expression for the Wiener optimal IIR filter can thus be written as 


HOE [25] (11.22) 


1— 2,27! 


where the gain G is given by 


Aan 9 
oe IT.23 
Ce Fol lel (11.23) 


It is interesting to note that the filter places a zero at z = y in an attempt to cancel the 
additive noise. Also, it is important to note that z, is just a function of the constant C, 
and, therefore, through (II.10), it is just a function of the input signal-to-noise ratio pin, 


the signal parameter a, and the noise parameter 7. 
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The following figures show the behavior of the pole z, for different situations. In 
Figure 4, the input signal-to-noise ratio (p;,) is fixed, but the signal parameter a varies 
from —1 to +1. We have plotted the location of the pole z, of the optimal filter for 
different values of y (y = 0 represents the white noise case). Notice that when a = ¥, the 
filter places the pole at the same location (z; = y), since it cannot distinguish the signal 
form the noise. Also observe that when a = +1, the pole is placed at z = +1, regardless 


of the value of y. In Figure 5, the noise parameter 7 has a fixed value of 0 corresponding 
1 

0.8 

0.6 

0.4 


0.2 


Pole of the JR filter 





—1 0.8 -06 -04 #-0.2 0 0.2 0.4 0.6 0.8 1 
signal correlation coefficient (alpha) 


Figure 4. Location of Pole of the IIR Filter as a Function of Signal and Noise Correlation 
Coefficients (@ and y) for Input Signal-to-Noise Ratio of 0 dB. 


to white noise, and the pole z, is plotted for different values of the input signal-to-noise 
ratio. Notice in Figure 5 that when the input SNR is small, the pole of the IIR filter 


approaches the signal parameter a. ‘This can be shown analytically as follows. 
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input signal-to-noise ratio 


Pole of the IIR filter 





—1 -08 --06 -04 =-0.2 0 0.2 0.4 0.6 0.8 1 
signal correlation coefficient (alpha) 


Figure 5. Location of Pole of the IIR Filter as a Function of a for y = 0 and Different 
Values of the Input Signal-to-Noise Ratio. 


First, from (II.10) we see that 





1 2 

lim (C) = ano 

Pin 0 

Thus, 
— 1f(1+e?%) (1 + a2)? — 40? 
ae (Gee ar 
1 
= 55 (20°) cy (II.24) 


In a similar manner, we can show that for a large input SNR, the pole approaches the 


noise parameter yy. We have 


2 
lie Ce ars 


Pin 70CO a 
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Figure 6. Location of the Pole of the IIR Filter as a Function of a for y = —0.5 and 
Different Values of the Input Signal-to-Noise Ratio. 


Therefore 
2 Q\2 _ An? 
‘ee D 3 (C2. (+7) "| 
pin—0o 2 ry 7? 
1 
On ee 11.25 
a 27) y (II.25) 


This characteristic can also be observed in Figure 6, where we have plotted the location 
of the IIR filter pole as a function of a, for different values of the input signal-to-noise 
ratio, and for y = —0.5. As we will see in a later section, this behavior leads to a better 
performance, in terms of the processing gain, for smaller values of the input signal-to- 


noise ratio. 


iw 


B. PROCESSING GAIN FOR THE IIR FILTER 
Using the first order signal and noise model, we can derive an analytical expression 
for the processing gain (1.29) for the IIR Wiener Filter. For this we first need the input 
and output signal-to-noise ratios. The input signal-to-noise ratio is given by (1.30) or 
(II.11), while the output signal-to-noise ratio is given by (1.31). Hence, the problem is 
to find expressions for R,,(0) and R,,(0) in terms of a, y, 03, and 02. To do this, first 
observe that the complex spectral density of the output of a linear system is given by 
[Ref. 5] 
Sa — Biz) Ai GaSe) (II.26) 


where H(z) is the filter transfer function and S, is the complex spectral density function 
of the input. Since the correlation function is related to the complex spectral density 


function by the inversion formula 


Ry (I) 


i 1 
on] $ Dull) 2 9 9ae 
S- Residues[S,(z)z'~"] 


where the residues correspond only to the poles within the unit circle, we have 


R, (0) \” Residues[S,(z)z~"] 


\_ Residues[H(z)H(z~")S,(z)z7*] (i278) 


To evaluate R,,(0) we use this formula assuming that the input complex spectral density 


function S,(z) is that of the signal. Thus from (11.26), (1.22), and (II.22) we obtain 


1 1--yz o*(1 — a?) 





2 a 
I — a or , 
Sye(2)z a zl—2271 l-—zz (1—az—!)(1 —az) ee 
and, according to (II.27) we have 
R,,(0) = RES,=2, + RES,9 + RES,<o (1.29) 


where the three residues are given by 


_ @0=-w d= z)os(-e*) 7, , 
ee (z —2z,)(1—22z)(1 —az)(1—- =n) y 


ZZ 
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G?o;(1 — a*)(1 — y¥/a)(1 — ya) 
(1 — z?)(1 — a/z)(1 — a2) 


G2(1 — yz71)(1 — yz2)02(1 — 2) 








RESisa = (1 — z27-!)(1 — z,z)(1 — az)(z — a) Za) on 
GaGa aaa elie a) 
(1 = z Ja) = 21a) 
G?(1 — yz7)(1 — yz)o2(1 — a?)z ms 2 
ae z(1—227)(1-—zz)1 -—az)1- a) ¥ ne : 
Thus combining these results, we have 
_ ag? [= aW/a)G =a) a") | (L~7/e). = 7a) | 
Fey,(0) - oe fF = Ziel = a/z)(1 = 0:21) (1 os z/a)(1 — Za) es) 


In a similar manner we see that the output complex spectral density due to the noise 


alone is 
(=325° oe a 
oa C= Dey olay) Y) 
d 1-227! l—zz (l-yz7!)(1- yz) 
e Co 1-7) 


"daz \—a2) —_— 


Applying (II.27) and evaluating the expression at the single pole z = z, then leads to 


1 
0=c2 LZ 
Ry, ( ) Fy, (1 — z?) (I 3 ) 
By taking the ratio of (II.30) and (II.32) we compute the output signal-to-noise 


ratio 
go Be sa — 24) 
""—— (l-a/a)(l-—az) (1—2z/a)(1— 2a)(1 — a?) 
Finally, by substituting (II.11) and (11.33) into the definition (1.29) we obtain the final 


(11.33) 


expression for the processing gain as 





Figure 7 shows the theoretical and experimental values of the processing gain for 


different values of a, and for y = 0, representing additive white noise, at a signal-to-noise 


ae 


ratio of 0 dB. Figure 8 shows the same curves, but for y = —0.5 (colored noise) at an 
input signal-to-noise ratio of —10 dB. The experimental curve was obtained by averaging 
measured results for 10 different trials for each value of a, in order to obtain a smaller 


curve variance. Notice that there is very good agreement between the theoretical and 


experimental values. 


—— theoretical 
* experimental 


processing gain (dB) 
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Figure 7. Comparison between Theoretical and Experimental Values of Processing Gain 
for the IIR Wiener Filter as a Function ot a, for 7y = 0 (White Noise) and mp Signal- 
to-Noise Ratio 0 dB. 


Observe from Figures 7 and 8 that the processing gain for the first order IIR filter 
has its maximum values at a = +1. We can examine the behavior explicitly at these 


limits as follows. For a — +1 the constant C in (JI.10) approaches +2. It follows then 
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——— theoretical! 
- experimental 
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Figure 8. Comparison between Theoretical and Experimental Values of Processing Gain 
for the IIR Wiener Filter as a Function of a, for y = —0.5 and Input Signal-to-Noise 
Ratio —10 dB. 


from (II.7) and (JI.8) that the root z,; approaches +1. Using this result in (II.34) we find 


lime++1PG = QQ-yd-yG=-24) 1 & fa) a) eae 


ipl 2h) eee 2 79) lee eS Zyl ay) ley leer) 


let 


7c 
@1.35) 


Figures 9 through 11, show the behavior of the processing gain for different values 
of the input SNR (p;,), and for y = 0, y = —.5, and y = —.999. Notice that in all 
of these cases the processing gain approaches infinity as a > +1. Intuitively, we can 
see from (IJ.22) that, for a completely correlated signal (a = +1), the optimal IIR filter 
attempts to null the noise (by placing a zero at y) while completely let the signal pass 


(pole at z} = a = +1). Thus, while the input signal-to-noise ratio remains finite, the 
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output signal-to-noise ratio and thus the processing gain becomes infinite. 


Input signal—to—noise ratio 


processing gain (dB) 





Figure 9. Processing Gain for the IIR Wiener Filter as a Function of a for y = 0 and 
Different Values of Input Signal-to-Noise Ratio. 


C. MEAN SQUARE ERROR FOR THE IIR FILTER | 

An analytical expression for the mean-square error of the Wiener filter for a first 
order AR signal in white noise is given in [Ref. 5]. Here we extend that result to the 
general colored noise case and derive an analytical expression for the MSE given by 
equation (1.33). 


If the error is defined as e(n) = s(n) — 8(n), then (1.33) can be written as 


o2 
MSE = —§ (11.36) 


2 
Os 
where o? is the unnormalized mean-square error or error variance. It is shown in (Ref. 


5] that the complex cross spectral density function between the signal and the error is 
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Figure 10. Processing Gain for the ITR Wiener Filter as a Function of a for y = —0.5 
and Different Values of Input Signal-to-Noise Ratio. 





given by 
S22) = onal — ize) (ise 
Substituting (1.22) and (II.22) then yields 
o*(1 — a?) 1- yz 
Sul®) = ae) (1 -G— ) (11.38) 


By the orthogonality theorem of linear mean-square estimation, the error variance @? is 


equal to the cross-correlation function R,,(l) evaluated at lag 1 = 0. Therefore from the 


inversion formula for the complex cross spectral density we have 
1 = 
Ga ieee a7 f Sx(2)2 tdz (11.39) 
This integral, involving (II.38) can be evaluated by residues, giving 


ao? = YS Residues[S,.(z)z~*] 
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Figure 11. Processing Gain for the IIR Wiener Filter as a Function of a for y = —.999 
and Different Values of Input Signal-to-Noise Ratio. 





o7(1-— a?) 1- yz 
eS c—~*) “li, _ 
(1 — az)(1 — az-) ( l— 22 eo pee 
1— 21a 
Finally, using this result in (11.36) we obtain 
ae 
vice) “eee e) (11.40) 


(1 — za) 


Figure 12 compares the theoretical mean-square error for y = 0 and p;, = 0 dB 
with the averaged measured squared errors for the same case. Again, there is a good 
agreement between the theoretical and the experimental results. Figure 13 shows the 
behavior of the mean-square error for input signal-to-noise ratio of —10 dB, 0 dB, and 


10 dB. Notice that, when a = y = 0, MSE = o?/(05 + 07). For example, when p;, = 0 
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Figure 12. Comparison between Theoretical and Experimental Values of Mean Square 
Error for the IIR Wiener Filter as a Function of a for y = 0 (White Noise) and Input 
Signal-to-Noise Ratio of 0 dB. 


dB, the power of the signal equals the power of the noise, and MSE = 0.5, as expected. 
The same analysis for p;, = 10 dB results in MSE = 0.1/ ee 

Figure 14 shows a case atten the noise is not white (y = —0.5). The results are 
qualitatively similar to those for the white noise case. However the curves are skewed, 
indicating that the worse performance in terms of mean-square error occurs when the 


signal has a correlation coefficient equal to that of the noise. 


D. SIGNAL DISTORTION FOR THE IIR FILTER 


In this section we derive an explicit expression for the signal distortion (1.34) 
produced by the Wiener filter for the problem of a signal in additive noise. As in the 


previous sections, the signal and the noise are each represented by a first order AR model. 
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Figure 13. Mean Square Error for the IJR Wiener Filter as a Function of a for 7 = 0 
(White Noise) and Different Values of Input Signal-to-Noise Ratio. 


To evaluate the term R,,,(0) in (1.34), we begin with the cross complex spectral 


density function between the input and output of the filter is given [Ref. 5] by 
Sey, (2) = H(2)S,(2) 
Therefore, as before, we can evaluate R,,,(0) from 


Rsy,(0) = 5_ Residues[Sy,(z)2~*] 





1--yz o2(1 — a?)z7! 
3 d ) poke Al 
Pe ss sobacat (; —zz/ (1—az)(1 - az!) oe 
The only residue is given by 
_ (l-—-yz) , (1 — a”) 
Resid cella (1—2z) °(l-—az)(1 — az)2z-!(z -— a) — 
= fe (11.42) 


*~ (1 - za) 
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Figure 14. Mean Square Error for the IIR Wiener Filter as a Function of a for y = —0.5 
(Colored Noise) and Different Values of Input Signal-to-Noise Ratio. 


resulting in 

(1 — ya) 
ee a Za) 
Then, by substituting (11.43), (II.30), and R,(0) = o? into (1.34) we can obtain an 


ren (0) Sa26 (11.43) 


expression for signal distortion, | 
E 
=l|-— [.44 
SD = (11.44) 


where 


ie a (11.45) 


F = oe eS (1 — y/a)(1 — ya) 
ere one aye) z/a)(1 — 21a) 


Figure 15 shows the theoretical expression (11.44) against the measured results as 


(11.46) 


a function of the correlation factor a, for white noise, with an input signal-to-noise ratio 


29 


of 0 dB. Notice that for values of @ close to +1, 1.e., for highly correlated signals, the 


distortion is very low. This is also apparent from Figure 16 when the signal distortion 
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Figure 15. Comparison between Theoretical and Experimental Values of Signal Distortion 
for the IIR Wiener Filter as a Function of a for y = 0 and Input Signal-to-Noise Ratio 
0 dB. 


is plotted for several different values of the input signal-to-noise ratio. Figure 16 also 
shows that the signal distortion is lower for higher values of the input signal-to-noise 
ratio, because the filter is able to pass more of the signal. Since we already know that 
lima-+1 21 = 1, we can evaluate the limg-,; for (II.46) and verify that lim,.,; SD = 0. 
Also, for a = y (which implies that H(z) is an all-pass filter) the distortion is zero, as 
expected. However, there is a value for a, which we shall call the critical value a,, where 


the distortion reaches its maximum value. This value depends on jp. 
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Figure 16. Signal Distortion for the IIR Wiener Filter as a Function of a for y = 0 
(White Noise) and Different Values of Input Signal-to-Noise Ratio. 


Figure 17 shows the result for y = —0.5. Again the distortion peaks at specific 
critical values of the correlation coefficient a,. However, in this case the two values do 


not occur at locations symmetrically distributed about the origin. 


FE. SUMMARY 

The analysis of this chapter based on the first order AR signal and noise model 
shows that the optimal filter for estimating the signal is also of first order. The filter has 
a zero at the location y corresponding to the noise correlation coefficient, and a pole at 
location z; which moves from —1 to +1 as the signal correlation coefficient a moves from 
—1 to +1 (see Figure 5). For low values of the input signal-to-noise ratio the pole z, 
becomes very nearly equal to a, while for high input signal-to-noise ratio z} approaches 


y and the filter becomes an all-pass filter. 
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Figure 17. Signal Distortion for the IIR Wiener Filter as a Function of a for y = —0.5 
(Colored Noise) and Different Values of Input Signal-to-Noise Ratio. 


The processing gain (PG) for the optimal filter increases with increasing signal 
correlation and becomes infinity in the limit as a approaches +1 (perfectly correlated 
signal). The processing gain decreases as the input signal-to-noise ratio increases indi- 
cating that less improvement is possible when the input signal-to-noise ratio is already 
high. 

The normalized mean-square error (MSE) for the optimal filter decreases with 
increasing correlation and becomes zero in the limit as a approaches +1. It is fairly flat 
for most values of a and decreases most rapidly as a approaches +1. The normalized 
mean-square error (MSE) also increases when the input signal-to-noise ratio decreases. 

The signal distortion (SD) becomes zero when a@ = 7, where the filter becomes an 


all-pass filter, and when a@ approaches +1, where the signal is completely correlated. 
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It achieves its maximum values for moderately high values of correlation. In the 
case of white noise this occurs around 0.8 < |a,| < 0.9. (see Figure 16). 

The performance of the filter for colored noise parallels that of the filter for white 
noise. However, the symmetry with respect to a = 0 is lost. When a = 7, the processing 
gain goes to zero, the mean-square error achieves its maximum value, and the signal 
distortion goes to zero. Maximum values of the signal distortion occur at two distinct 


points above and below the value of the noise correlation parameter y (see Figure 17). 
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III. ANALYSIS OF THE FIR WIENER FILTER 


In this chapter we develop expressions for the FIR form of the Wiener filter for 
the case where the signal and the noise are, again, each represented by a first order AR 
model (equations (1.19) and (I.24)). The chapter is divided into two parts. In Part A, we 
derive analytical expressions for the first order FIR Wiener filter, i.e., a FIR filter with 
only two coefficients. In Part B, we use a more general approach to derive expressions 
for a filter of any order. Although Part A treats a restricted particular case, it allows us 
to obtain simple expressions for the measures of performance as functions of the model 
parameters. On the other hand, the results of Part B permit us to plot the measures of 


performance for more general cases, where the FIR filter can be of any order. 


A. THE FIRST ORDER FIR WIENER FILTER 

In Chapter I we introduced the matrix form of the Wiener-Hopf equation (I.16) 
for the FIR Wiener filter and the associated mean-square error (I.17). For the first 
order FIR Wiener filter, the vector of filter coefficients h has only two elements, i.e., 
h = [h(0) h(1)]}?. Thus we can obtain expressions for these two coefficients by solving 
equation (I.16) as follows. Since the noise is assumed uncorrelated with the signal, R,(l) 
and R,,(l) are given by (1.3) and (1.4). Evaluating (1.20) and (1.25) for lag / = 0 and 1, 
and substituting in (1.3) yields 


R,(0) = of + 0% = oF[1 + pin] cities 


and 


R,(1) = ofa + ony = of[y + apin| (ine?) 


Similarly, from (1.20) and (1.4) we obtain 


Rsz(0) = 0? (III.3) 


Ss 
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and 


Fee waar (III.4) 


Using (III.1) through (III.4) in equation (1.16) results in 


Ao) |_| 2 
Aly, | Mee ey 


Then, solving this matrix equation by inverting the matrix, we can express the filter 


(Pin ES 1) (Pin 5 7) 
(opin +7) (Pin + 1) 


oe 











coefficients as 


h(0) s Pin Pin + 1 —(QPin ate 7) 1 
h(1) (Pin + 1)? — (pin + 7)? ~(apin +) ame wy 





or, 
a eee) 
h(O) = pin > III.5 
( ) A WOin a Ip) od (QPin ate aa ( 
(257) 
Jal) oh — ITI.6 
( a (Pin ra ie a (Pin a ae ( 
The transfer function for the filter is then given by 
H(z) = h(0) + A(1)z7" 
or 
H(z) = h(0)(1 — 2z7") | (III.7) 
where the zero of the filter is located at 
ge 
2=20 >= ~ A(0) (ITI.8) 
or 
= (III.9) 


(Pin gees QO Pin a ay) 
Figures 18 and 19 show the location of the zero 2 of the first order FIR Wiener 
filter as a function of @ for different values of y, and for an input signal-to-noise ratio of 0 


dB and —10 dB, respectively. From these figures we can identify three interesting cases. 
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Figure 18. Location of the Zero of the First Order FIR as a Function of the Signal and 
Noise Correlation Coefficients (a and y) for Input Signal-to-Noise Ratio of 0 dB. 


First, for a = +1 and y # £1, and any value of the input signal-to-noise ratio, the filter 
places the zero at z = +1. This can be easily verified from equation (III.9). Secondly, 
when @ = +1 and y = +1 we see that z) assumes different values. These values can be 
found by setting -y = +1 in (III.9) obtaining 


+1—a 


a a a 
Pint Va Pin + Oo 
and evaluating 


Ste ney —— Sel 


7 eT ee 
© ae] Oe O Din Fa etl —2apin F1 2Pin +1 


(III.10) 


Finally, when a = y # +1, the filter places the zero at z = 0, regardless the value of 
input signal-to-noise ratio. This seems intuitively correct, since when @ = 7¥ the filter 
cannot distinguish the signal from the noise, and should become an “all pass” filter 


(Zo = QO). 
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Figure 19. Location of the Zero of the First Order FIR as a Function of the Signal and 
Noise Correlation Coefficients (a and y) for Input Signal-to-Noise Ratio of —10 dB. 


Figure 20 shows the location of the zero of the first order FIR filter for y = —0.5, 
and for different values of the input signal-to-noise ratio. ‘The three cases discussed above 


are again evident from this figure. 


1. Processing Gain for First Order Filter 
To obtain an analytical expression for the processing gain defined in (1.29), we 
first find an expression for the output signal-to-noise ratio (1.31) of the first order FIR 
filter. Starting with the general expression [Ref. 5] 
R,QO= >> Dd) Rell — ky + ko)h(ki)h(ko) 
kg=—00 ky} =—00 


and assuming that the signal alone is applied to the filter we have 


1 1 


Ry(0) = >> >> Rs(—ki + ko) h(ki)A(Ko) 


kg=0 k,=0 
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Figure 20. Location of the Zero of the First Order FIR Filter as a Function of a for 
y = —0.5, and for Different Values of the Input Signal-to-Noise Ratio. 


= vet (MOD 43] +28e) 





and by applying (III.8) we obtain 


R,, (0) = h?(0)o?(z§ — 2200+ 1) (III.11) 
By an identical procedure, we obtain 
R,, (0) = h*(0)07(z§ — 2207 + 1) (III.12) 


Finally, by using (III.11) and (III.12) in conjunction with (II.11), (1.29) and (1.31), we 


can express the processing gain for the first order FIR filter as 


a (26 + 1 — 2z9a) 


[hii 
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Figure 21. Comparison between the Theoretical and Experimental Values of the Process- 
ing Gain for the First Order FIR Filter for y = 0 (White Noise) and Input Signal-to-Noise 
Ratio (Pin) of 0 dB. 


Figures 21 and 22 show that there is good agreement between the theoretical and 
the experimental values of the processing gain of the first order FIR filter, for the case 
when y = 0 and pi, = 0 dB, and when y = 0.5 and p;, = —10 dB, respectively. Figure 
23 shows that the processing gain is independent of the input signal-to-noise ratio at 
three different points. The first point eects when @ = 7, aS mentioned before, when the 
filter becomes an “all pass” filter; here the resulting processing gain is 0 dB. The other 
two points occur when a = +1 and y 4 +1, where zp = $1, and upon substituting this 
result into (III.13), we find that the processing gain becomes PG|,-+) = 2/(1+7), ie., 
independent of the input signal-to-noise ratio. 

Figure 23 also shows the general behavior of the processing gain for the first 


order FIR filter in the white noise case, for different values input signal-to-noise ratio. 
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Figure 22. Comparison between the Theoretical and Experimental Values of the Pro- 
cessing Gain for the First Order FIR Filter for y = 0.5 (Colored Noise) and Input 
Signal-to-Noise Ratio (p:,) of —10 dB. 


Notice that higher values of the input signal-to-noise ratio results in lower values of the 
processing gain. Intuitively, this happens because when the input signal s(n) is strong 
compared to the noise, the filter cannot make a large improvement. In the limit, as 


Pin —> co the processing gain goes to zero (except at a = +1). 


2. Mean Square Error for First Order Filter 
The (unnormalized) mean-square error can be computed from (I.14) using (III.3), 


(III.4), (III.5) and (III.6) yielding 


a2 = o — [o2h(0) + a07A(1)] 
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Figure 23. Processing Gain for the First Order FIR Filter as a Function of a, for y = 0 
(White Noise), and Different Input Signal-to-Noise Ratio. 


Alternatively, by applying (III.8), it can be written as a function of Zp: 


o? = 07(1 — h(0){1 — azo (III.15) 


Using (III.15) in (1.33) gives the normalized mean-square error as 
MSE = [1 — h(0){1 — azo] (III.16) 


Figures 24 and 25 compare the theoretical and experimental values of the nor- 
malized mean-square error for different values of input signal-to-noise ratio and the noise 
correlation parameter y. Again, there is a good agreement between the theoretical and 
the experimental curves. Figure 26 shows the behavior of the normalized mean-square 
error for different values of input signal-to-noise ratio. We see that for higher values of 


the input signal-to-noise ratio the mean-square error assumes lower values, as expected. 
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Figure 24. Comparison between the Theoretical and Experimental Values of the Mean- 
Square Error for the First Order FIR Filter for - = 0 (White Noise) and Input Signal- 
to-Noise Ratio of 0 dB. 


3. Signal Distortion for First Order Filter 
The signal distortion defined by (1.34) is easily evaluated. It is easy to show that 
the cross correlation between input and output of a linear system is related to the input 


autocorrelation function by the convolution relation (Ref. 5] 


Rey(l) = > h(k) Rell) (111.17) 


k=—oo 


Hence, for our first order filter we have 


Rey(0) = 2, h({k) Ra(-k) 
= 9;(h(0) + ah(1)) 
= o*h(0)(1 — az) (III.18) 
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Figure 25. Comparison between the Theoretical and Experimental Values of Mean- 
Square Error for the First Order FIR Filter as Function of a for y = 0.5 (Colored Noise) 
and Input Signal-to-Noise Ratio of —10 dB. 


Then, by applying (III.18) and (III.11) to (1.34), we obtain 


Sp = z(1 — a?) 


cali (III.19) 


Figure 27 shows the theoretical and experimental values of the signal distortion 
for different values of a, and for y = 0 (white noise) at a input signal-to-noise ratio of 0 
dB. We again observe good agreement between the experimental and theoretical results. 
As in the case of the IIR filter, the distortion is zero for a = y and a = +1, and achieves 
a maximum value at a point 0 < a, < 1. In Figures 28 and 29, the noise parameter 
7 has fixed values of 0 (white noise) and 0.5, respectively, and the signal distortion is 
plotted for different values of the input signal-to-noise ratio. Notice that for lower values 


of input signal-to-noise ratio the signal distortion assumes higher values. In addition, 
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Figure 26. Mean-Square Error for the First Order FIR Filter as a Function of a for 
y = 0.5 (Colored Noise) for Different Values of Input Signal-to-Noise Ratio. 


notice that, regardless of the input signal-to-noise ratio, for a = y or for a = +1, the 


signal distortion is zero. This can also be easily verified from (III.19). 


B. PERFORMANCE MEASURES FOR HIGHER ORDER 
FIR WIENER FILTERS 


For FIR filters of higher than first order (P > 2), it becomes extremely difficult 
or impossible to derive analytical expressions for the filter coefficients and zeros of the 
filter in term of the basic parameters a, y, and pin. However it is still useful to be 
able to compute theoretical values for the processing gain, mean-square error, and signal 
distortion in these cases. Therefore, in this section we develop a different approach to 
derive expressions for these performance measures, which can be plotted as a function 


of the various parameters. With these results, we can compare the performance the FIR 
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Figure 27. Comparison between the Theoretical and Experimental Values of Signal Dis- 
tortion for Different Values of a and for y = 0 (White Noise) at a Input Signal-to-Noise 
Ratio of 0 dB. 


filter of any order to that of the optimal IIR Wiener filter when they are applied to the 


first order signal and noise models of equations (1.19) and (1.24). 


1. Processing Gain for Higher Order FIR Filter 


The output power of a filter expressed in the form (1.15) can be written as 
E {y?(n)} 

= Ej{(h*x) (x*h)} 

= h’E{x x7}h 


R;(0) 


= h’R,h (III.20) 
Therefore, the output power when the signal and noise act alone is given by 


R,,(0) =h7R,h (III.21) 
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Figure 28. Signal Distortion for the First Order FIR Filter as a Function of a for y = 0 
and Different Values of the Input Signal-to-Noise Ratio 


and 
R,, (0) = h*R,h (iie225 
where R, and R, are the signal and the noise correlation matrices, which have the form 
o* o70: cee o2qlP-1 or On) swiss g2ylPoM 
2|P-2 P-2 
R. o7a: oe eee o2a! | R, = ony or eee Py! | 
o2qlP-1 eo a ees Ge | gaye oa ess a | 
(III.23) 
Thus from (III.21), (III.22), (1.29) and (1.31) we obtain 
h’?R,h 1 
PC = ee. mL 
G b7Rh p;, (III.24) 
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Figure 29. Signal Distortion for the First Order FIR Filter as Function of @ for y = 0.5 
and Different Values of the Input Signal-to-Noise Ratio 


The processing gain produced by the Wiener filter can be obtained by substituting the 
solution to the Wiener-Hopf equation (1.16) into (III.24). Specifically, from (I.16) and 
(1.18) we have 

h=(R,+R,)7'F; (III.25) 


Since we have all the necessary expressions, we can plot the values assumed by the 
processing gain (III.24) for different values of a, y, and pin. 

Figure 30 shows the behavior of the processing gain as a function of a for y = 0 
(white noise) and for different values of input signal-to-noise ratio. Notice that the 
processing gain has different maxima for a = +1, depending on the order of the filter. 


These maximum values can be easily predicted for the white noise case (y = Q), as 
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follows. For a = 1 and y = 0 the signal and noise correlation matrices (III.23) become 


geno. oe a, 0 0 
Gn 0 ar men AG) 
Rs={| | | R,=] . |. (III.26) 
Ga GO, ** aN fvO 0 = ia |) 
Under these conditions (III.25) shows that h(0) = h(1) =... = h(P —1). By applying 


these results to (III.21) and (III.22), we obtain 

h Rehi= Paeete (0) 
and 

h7R,h = 07 P h?(0) 
Finally, substituting these expressions into (III.24) results in 


ania 


A similar analysis shows that PG = P also for a = —1. 

This result shows that unlike the IIR filter, the processing gain for an FIR has 
a finite bound, which is equal to the filter length. This result can be easily checked in 
Figure 30.' Notice also that in general, for any value of a, larger length filters produce 


increased processing gain. 


2. Mean Square Error for Higher Order FIR Filter 
The normalized mean-square error (1.33) for any FIR filter is not difficult to obtain. 


Using (1.15) and (I.1) we can write 


E {(s(n) — h* 8 —h®* 7)(s(n) — 8" h—7* h)} 
o? 2h’ r,+h™ R,h+h’R,h 


E {(s(n) — y(n))*$ 


1Note that the vertical scale in Figure 30 is in dB 
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Figure 30. Processing Gain for the FIR Filter of Length P for y = 0 (White Noise) and 
Input Signal-to-Noise Ratio of 0 dB 


where the cross terms disapear because the signal and noise are uncorrelated. Then, 
substituting this expression in (1.33) we obtain 


2h? #,—-h7 R,h—h?R,h 


MSE = 1 - : 
Os 


(111.27) 


The result (II1.27) applies to any FIR filter. For the Wiener filter we can simplify 
the expression by noting from (1.16) and (1.18) that 


(R,+R,)h—r, =0 


Therefore, for the Wiener filter (III.27) simplifies to 


he ~ 
hers 
Z 
Os 





MSE = 1- (111.28) 
This result can also be obtained directly using (1.17). 
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Again, we have all the necessary elements to plot the normalized mean-square 
error as a function of the model parameters and the input signal-to-noise ratio. Figure 
31 shows the normalized mean-square error plotted as a function of a, for y = 0 and input 
signal-to-noise ratio of 0 dB. It can be seen that higher order filters result in lower values 
of mean-square error, as expected. However, for the longer filters, there is significant 
improvement in performance only for very high values of the signal correlation parameter 
a, i.e., the ability to reduce mean-square error depends heavily on the correlation present 


in the signal. 
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Figure 31. Mean-Square Error for the FIR Filter of Length P for y = 0 (White Noise) 
and Input Signal-to-Noise Ratio of 0 dB 


3. Signal Distortion for Higher Order FIR Filter 
To derive an expression for the signal distortion for the FIR filter of order P, we 


write the response of the filter for the signal alone using (1.15) as y,(n) = h? 8. Thus, 


ol 


we have 


E (s(n)aehn)} = E fe(n wha? Shah" aly | (III.29) 


and 


E {y,(n)?} = E {h” 8 87 h} =h" R, h (III.30) 
Substituting these expressions into equation (I.34) then produces the desired expression 


(hea. )7 


aS eS? 


(III.31) 


This can be evaluated for the Wiener filter by substituting the solution (III.25) for h. 

Figure 32 shows the behavior of the signal distortion for the FIR filter of different 
orders P, for y = 0 and input signal-to-noise ratio of 0 dB. The behavior of this measure 
is similar to its behavior in the case of the IIR filter. The signal distortion is 0 dB 
for a = +1 and for a = y and achieves a maximum value around a value a, ~ +0.8, 
depending on the length of the filter. In addition, the value of the signal distortion is 
greater for larger filter lengths. 


C. SUMMARY 

The analysis carried out in this chapter for the FIR Wiener filter shows that this 
filter has performance qualitatively similar to that of the IIR filter, although for the case 
of a first order AR signal and noise model the performance, in terms of the mean-square 
error and processing gain, is never quite as good. 

For the first order FIR filter, which has a single zero, the location of the zero 
changes from +1 to —1 as the signal correlation coefficient increases from —1 to +1. 
When the signal and noise have the same correlation coefficient (a = y) the zero moves 
to the origin (resultind in an all-pass filter); when both signal and noise are perfectly 
correlated (a = y = +1) the zero moves to a location that depends on the input signal- 
to-noise ratio (see (III.10)). This results in a steady state estimate for the signal which 


is equal to a constant fraction of the input observation sequence. 
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Figure 32. Signal Distortion for the FIR Filter of Length P for y = 0 (White Noise) and 
Input Signal-to-Noise Ratio of 0 dB 


The performance measures for the FIR filter approaches those of the IIR filter as 
the order of the filter increases. Figures 30, 31, and 32 show plots of the three performance 
measures for the FIR Wiener filter of orders 2, 4, and 10. These performances can be 
compared to those for the IIR Wiener filter in Figures 33, 34, and 35. A significant 
difference between the two types of filters is that the processing gain of the FIR filter, 
when the input signal is perfectly correlated is not infinite, but is limited to a value P 
(the filter length). Also the mean-square error does not go to zero when a = +1 as in 
the IIR filter case. However, the signal distortion for the FIR filter does go to zero for 
a= vy and a= +1, as in the IIR filter, and it is not as large as for the IIR filter. 
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Figure 33. Processing Gain for the IIR Filter for y = 0 (White Noise) and Input Signal- 
to-Noise Ratio of 0 dB 
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Figure 34. Normalized Mean-Square Error for the IIR Filter for y = 0 (White Noise) 
and Input Signal-to-Noise Ratio of 0 dB 
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Figure 35. Signal Distortion for the IIR Filter for y = 0 (White Noise) and Input Signal- 
to-Noise Ratio of 0 dB 
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IV. THE EXTENDED OPTIMAL FILTER 


We can conclude from the results of the previous chapters that the Wiener filter, 
although optimal in the mean-square error sense, is not necessarily optimal with regard 
to minimizing the signal distortion. This chapter addresses the possibility to improve 
the signal distortion performance of the filter, at the expense of increasing the output 
residual noise. In other words, we may be able to improve the signal intelligibility with 
some slight increase in mean-square error due to noise. 

Ephraim and Van Trees [Ref. 7] have suggested a more general filter for speech 
enhancement, based on one measure of the signal distortion; here we have adapted this 
idea using the signal distortion measure defined in Chapter I. This chapter formulates 
this more general problem and derives the solution using an FIR filter. It then provides 
an analysis of the performance of the generalized FIR filter for the first order AR signal 
and noise model, and finally suggests how the generalized filter may be used to obtain 


an appropriate trade-off between signal distortion and mean-square error. 


A. THE EXTENDED OPTIMAL FILTER PROBLEM 
We begin with the filter depicted in Figure 3, and the responses y,(n) and y,(n) 
due to the filter acting on the signal and noise alone. The problem as suggested in [Ref. 


7] is to minimize the distortion while constraining the residual noise power to satisfy 


E{y2(n)} <0? ~ Siva) 


2 


where o* is some chosen level of residual noise power. In this chapter, we apply this 


procedure using the signal distortion measure defined in (1.34). In the context of an FIR 


filter, the problem can be formulated as to minimize (III.31) subject to the constraint 


E {y,(n)} = h™R,h < 0? (IV.2) 


of 


Let us first consider the procedure for minimizing the signal distortion without 
the constraint (IV.2). To do this it is necessary to maximize the term 


(h7 &,)2 bh £,F7h 
h7R,h  h7R,h eee 


appearing in (IJI.31). This can be done by constraining the denominator to be any 


constant 


h’R,h = const 
and maximizing the numerator. Thus, by forming the Lagrangian [Ref. 5, 8] 
£ =h’£,f7h + u(const — h?R,h) 


where yz is a Lagrange multiplier, and setting the gradient of the Lagrangian to zero we 


obtain 


Val = lar h=— 2 Reh=0 
This shows that h must satisfy 
rreh =p Roh (IV.4) 


i.e., h is a generalized eigenvector for the two matrices r, r? and R,. The only non-trivial 
solution to (IV.4) is given by 
P= Re i: 


which is equal to the all-pass filter h = [1 00...0]’. 


When we introduce the constraint (IV.2) we need to form a new Lagrangian 
£ = —(h’r,)F7h + (const — h7R,h) + (a2 — h? R, h) (IV.5) 
where pz, and po are Lagrange multipliers. Setting the gradient to zero then yields 
Vat =2r, fF; -2u,R,h-2 wR, h=0 


or 


08 


where \ = [l2/p. In this case, we see that the solution is given by 





(IV.6) 


with py, = f7(R,+AR,)‘r,. The parameter \ can be related to o? through the constraint 
(IV.1). In particular, substituting (IV.6) into (IV.1) with the equality yields 


r (R,+AR,)7' R, (R,;+AR,)7 fF; = 0? 
or 
trr, 77 R, (R,+AR,)~* =o 
The solution (IV.6) for the extended optimal filter is the same as that of a Wiener 
filter with modified noise power. Some particular special cases are of interest. For 
= 0 we obtain the all-pass, h = [1 0...0]” (as we have already noted), resulting in 


the maximum residual noise power 07 = o7 and zero signal distortion. For \ = 1 we 


have the Wiener filter which minimizes the residual noise power 0? = 02. By choosing 
an appropriate value 0 < A < 1 we obtain minimum distortion with residual noise 
2 2 2 
Or Sep C;,- 
In summary, by choosing a value for A we can gain more control over the signal 
distortion at the expense of increased residual noise’ and mean-square error. In the studies 


presented below, it is shown that this added flexibility has some definite advantages for 


noise filtering. 


B. PERFORMANCE OF THE GENERALIZED FILTER 

By using equation (IV.6) in (IJI.24) we can study the processing gain of the filter 
as a function of the parameter 4 and the signal correlation coefficient a. Figure 36 shows 
the results of this computation for y = 0 (white noise) and an input signal-to-noise ratio 
of 0 dB. Notice that the processing gain is lower for lower values of A, demonstrating 
that there is less noise removal for 4 < 1. 

We can also substitute (IV.6) into equation (IIJ.27) to obtain an expression for 


the mean-square error as a function of a and 4. Figure 37 shows these results for the 
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Figure 36. Processing Gain for the Generalized Wiener Filter as Function of a and for 
7 = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB. 


same case as Figure 36, 1.e., white noise and an input signal-to-noise ratio of 0 dB. Again, 
there is decrease in performance for values of A < 1, going from minimum mean-square 
error for \ = 1, which corresponds to the optimal Wiener filter, to maximum for A = 0, 
which corresponds to an all-pass filter, that does not remove any noise from the observed 
sequence. 

Finally, we can apply (IV.6) to (IJI-31) and plot signal distortion as a function 
of a and A. Figure 38 shows the results for the same case as before. Here we see that 
the signal distortion of the filter improves for higher values of A, going from maximum 
distortion for \ = 1 (Wiener filter) to minimum distortion for 4 = 0, where the resulting 
all-pass filter does not alter the signal. 

The rate of change of the measures of performance PG, MSE and SD as a func- 


tion of \ allows one to make useful trade-offs. Notice from Figures 36 through 38 that 
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Figure 37. Mean-Square Error for the Generalized Wiener Filter as Function of a and 2 
for y = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB 


for values of A greater than approximately 0.4 the performance of the filter in terms of 
processing gain and mean-square error does not change significantly, while the s signal 
distortion decreases significantly over this range of values of A. This implies that the gen- 
eralized filter with an appropriately chosen value of 4 may be more desirable in practical 
applications involving noise removal than the standard Wiener filter. Since the solution 
(IV.6) for the filter is equivalent to that of a Wiener filter with an addition parameter, it 
is convenient to adapt this filter for use in a short-time noise removal algorithm. We will 


see in the following chapter a practical application of the new algorithm, which provides 


good performance and more control over the results 
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Figure 38. Signal Distortion for the Generalized Wiener Filter as Function of a and A 
for y = 0 (White Noise) and Input Signal-to-Noise Ratio of 0 dB. 


ve APPLYING EXTENDED OPTIMAL 
FILTERING TO UNDERWATER SIGNALS 


A. INTRODUCTION 

In this chapter we apply the solution of the extended optimal filter expressed 
in (IV.6) to the short-time filtering algorithm developed by Frack in [Ref. 4]. This 
algorithm assumes that both the signal and the noise correlation functions (R,(J) and 
R,(l)) are unknown a priori; therefore they must be estimated from the data at hand. 
Since only z(n) = s(n) +7(n) is observed, however, only R,(l) can be estimated directly. 
Nevertheless the nature of the problem provides a way to compute the needed statistics. 
Since the signal is very short (on the order of seconds or milliseconds) compared to the 
time over which the noise statistics are likely to change, an estimate for R,(l) can be 
made from the received data prior to the onset of s(n). By using (1.3), this estimate can 
in theory be subtracted from R,(l) to produce an estimate of R,(J). 

Although there is no explicit analytic expression for the parameter A, we can 
obtain the value of \ corresponding to any desired residual noise level by solving (A). In 


the case of white noise, this is equivalent to writing (IV.2) as 
Gael Calla i —io- (V.1) 
and then ( by IV.6) solving for \ from 
2 
ce ot 
| (R. + AR) "Fs [P= (V.2) 
n 


Therefore, by setting the level of the residual noise o? that can be tolerated, we can 
assign values for \ in each segment by iterating on (V.2) until the equality is satisfied, 
thus obtaining the value of A that minimizes the signal distortion. Thus, in principle, all 
of the quantities to perform the extended optimal filtering over a short time interval can 
be computed, and this can be repeated over successive blocks of data. 

The estimate of R,(/) by subtraction of the estimated correlation functions is 


not well formed because there is no guarantee that such an estimate will be positive 
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Figure 39. Prewhitening in Short-Time Extended Filtering Algorithm. 


(semi)definite. This problem is mitigated if the noise is white because the procedure 
then involves subtraction of only a single parameter, the white noise variance, from the 
estimated R, at lag zero. Further, the estimate of this single parameter has lower variance 
than the estimate of the correlation function as a whole. Therefore before any further 
steps, the entire data set is processed by a linear predictive filter that whitens the noise. 
After noise removal, the data is processed by the inverse filter as shown in Figure 39. 
The prewhitened data is segmented into blocks where an estimate of the local cor- 
relation function R,(I) is formed for each segment. Also, for each segment, the algorithm 
finds the value of 4, since ) is a function of the R, (1), R,(l), and o?. Extended optimal 
filtering is then performed for each segment using a filter designed for the segment, and 
the data is processed by the inverse filter to undo the effects of the prewhitening. In 
performing the filtering, the data is processed both forward and backward through the 
extended optimal filter. (This gives an approximation to a symmetric noncausal filter of 
twice the length.) Since the optimal filters are different for each block, discontinuities at 
the boundaries can arise. The effect of such discontinuities can be minimized by using 
points from the adjacent segment to filter the eariy points of the current segment. In the 
algorithm, the data is actually processed twice. The data is first segmented and filtered 
and the resulting frames are weighted by a triangular window (see Figure 40). The data 
is then resegmented using frames shifted by half of the frame length, filtered again and 
weighted by a triangular window. The two weighted sets of data are then added to pro- 
duce the final result and minimize any effects that may occur at the boundaries between 


frames. Additional details of this processing can be found in [Ref. 4]. 
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Figure 40. Overlap Averaging Technique Used in Noise Removal. 


B. RESULTS 


To obtain a quantitative measure of the signal distortion, we: first need to have 
a clean signal s(n), and obtain y,(n), the response of the filter to this signal s(n), as 
explained in Chapter I. In this way we can demonstrate our conclusions about the im- 
provement in the signal distortion accomplished by the extended filtering algorithm. 

For these experiments, an observation sequence x(n) was generated by adding a 
white noise sequence with variance o2 to the clean signal s(n). The sequence x(n) was 
then processed using the algorithm described above to obtain the filter parameters for 
each segment of this sequence. These same filter parameters were then used to process 
the corresponding segments of the clean signal s(n) to obtain y,(n). 

We can now compare the effects of the filter on the signal with respect to signal 
distortion. In order to do that, we introduce the estimates 


SD = 1 _ (La s(n) w(n))? 


dn $*(n) Ln y3(n) a 


and 
TOD __ ey va y(n))? 
Me eee 


where the summation is over all points in all segments. 


(V.4) 


For all the following experiments, the segment (frame) length is 100 points, the 


filter order is 35, and the AR model order for the noise was arbitrarily set to 10 (this 


69 


parameter could be set to any value since the noise is white). The first 1,000 points data, 
consisting of noise, was used for the noise sequence. 

Figure 41 shows the result of filtering a clean synthetically generated short pulse 
signal with 1,400 points (41(a)), contaminated by white noise with variance o2 = 0.36 
(41(b)), using the traditional Wiener filter. The result, shown in Figure 41(c), indicates 
that the filter is effective in removing the noise. The signal distortion estimate (V.3) 
for this case is SD = 0.0358, and the normalized mean-square error estimate (V.4) is 
MSE = 0.2249. 

Now, by applying the extended filtering algorithm for the case where more resid- 
ual noise can be tolerated, we can obtain lower signal distortion. Figure 42 shows the 
graphical results for this case where the residual noise power is set to correspond to 30% 
of the input noise power (o7/07 = 0.30). Part (d) of the figure shows the value of A 
that resulted for 02/07 = 0.30. Notice that \ assumes values less than 1 for some of the 
segments. The signal distortion estimate for this case becomes SD = 0.0541, which is 
an improvement of about 3%, and MSE = 0.1910, a degradation of about 0.7%, when 
compared to the traditional Wiener filtering shown above. 

Figure 43 shows the results for an extreme case when the residual noise power is 
70% of the input noise power. Here we can see that A assumes very low values for most 
of the segments. The signal distortion estimation for this case is very low SD = .0358, 
decreasing the signal distortion by a factor of 55%, and MSE = 0.2249, a degradation of 
about 18%. 

The following table summarizes the results of these experiments using three dif- 
ferent settings for the ratio 02/07. The values in parenthesis indicate the increase or 


decrease from the values achieved by the Wiener filter. 


The results shown here demonstrate that when more residual noise can be toler- 
ated, the signal distortion produced by the filter decreases significantly while the mean- 
square error increases by only a modest amount. 


The noise removal algorithm was also applied to real underwater acoustic data. 
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Wiener ) ss Generalized Filters Filter 


Filter 


SD 0.0556 0.0541 2 0466 0.0358 
(—3%) —19%) (—55%) 

MSE | 0.1896 0.1910 0.1985 0.2249 
(+0.7%) (+5%) (+18%) 


Table I. Results from the Generalized Filter Experiments. 






Figures 44 and 45 shows the results of applying the extended filtering technique to data 
representing a killer whale song. In Figure 44 we used the traditional Wiener filtering 
process, while in Figure 45 we allowed the residual noise to be 20% of the input noise 
power. For this case, it is difficult to quantify the performance, since ae true underlying 
signal is unknown. However, upon listening to the signal, the results are at least com- 
parable to those of the Wiener filter and the slight increase in residual noise is the price 


paid for lower signal distortion. 
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Figure 41. Results of the Application of the Extended Optimal Filtering Technique to 
a Synthetically Generated Short Pulse Signal with Added Low Power White Noise, for 
Minimum Residual Noise (A = 1). (a) Original Clean Data. (b) Original Data plus 
White Noise. (c) Processed Data. 
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(d) 


Figure 42. Results of the Application of the Extended Optimal Filtering Technique to 
a Synthetically Generated Short Pulse Signal with Added Low Power White Noise, for 
Residual Noise Power Equivalent to 30% of the Input Noise Power. (a) Original Clean 


Data. (b) Original Data plus White Noise. (c) Processed Data (d) Values of A for Each 
Segment. 
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(d) 


Figure 43. Results of the Application of the Extended Optimal Filtering Technique to 
a Synthetically Generated Short Pulse Signal with Added Low Power White Noise, for 
Residual Noise Power Equivalent to 70% of the Input Noise Power. (a) Original Clean 
Data. (b) Original Data plus White Noise. (c) Processed Data (d) Values of A for Each 
Segment. 
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Figure 44. Results of the Application of the Extended Optimal Filtering Technique to a 
Killer Whale Song for Minimum Residual Noise (A = 1). (a) Original Noisy Data. (b) 
Processed Data. 
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Figure 45. Results of the Application of the Extended Optimal Filtering Technique to a 
Killer Whale Song for Residual Noise Power Equivalent to 20% of the Input Noise Power. 
(a) Original Noisy Data. (b) Processed Data. 
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VI. CONCLUSIONS 


A. THESIS SUMMARY 

The first three chapters of this thesis provide an extensive analysis of the IIR 
and FIR Wiener filters for removing additive noise from a desired signal. Both signal 
and noise are represented by first order autoregressive (AR) models. Three performance 
measures, namely processing gain, normalized mean-square error, and signal distortion 
are defined and evaluated for the Wiener filters based on these models. We believe that 
the behavior of these filters in terms of the defined performance measures is typical even 
for other cases with more general signal and noise models. 

It was seen that the FIR filter has performance similar to that of the IIR filter, 
although it does not have the same ideal limiting characteristics such as infinite process- 
ing gain when the signal becomes perfectly correlated. We also showed that while both 
forms of the filter provide lower mean-square error and higher processing gain as the 
signal becomes more correlated, signal distortion is not a monotonic function of correla- 
tion. In particular the signal distortion for both forms of the filter peaks in a region of 
moderately high correlation and increases when the input signal-to-noise ratio decreases. 
This distortion introduced by the filter may be undesirable in some applications. 

Chapters IV and V deal with a way to improve the signal distortion resulting 
from the Wiener filter by considering a more general filtering problem. The result is 
a generalized solution that can reduce the signal distortion of the Wiener filter, with 
some increase in the mean-square error and residual noise. This solution was adopted 
in a short-time filtering algorithm developed by Frack |[Ref. 4] and tested. The results 
illustrate that the extended filter provides more flexibility and allows more control over 


the signal distortion introduced by the filtering process. 
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B. SUGGESTIONS FOR FUTURE RESEARCH 

The results from this thesis can drive some further discussions, including the 
performance of aural tests to evaluate the performance of the extended optimal filtering 
technique when applied to real sonar data, and application to real-time signal processing. 

In the time set aside to perform this thesis research the major portion of the effort 
involved the analysis of the Wiener filter and its generalization. Relatively little time 
was left for considering implementation details and more extensive testing on real data. 
For more efficient implementation, the nonlinear equations that specify the parameter \ 
in terms of the residual noise o? need to be inverted more efficiently. This could be done 
by a more effective iterative method or by inverting the function at a number of points 
and performing table look-ups. 

While the new algorithm is known to permit a trade-off in signal distortion versus 
mean-square error, the practical effects of this trade-off have not been truly evaluated. 
For example, formal listening tests have not been conducted to see (over some class 
of signals) if listeners preferred the results of the generalized filter over those of the 
conventional Wiener filter or, indeed, if they could hear the difference. The greatest 
benefits may come in the use of this new algorithm in conjunction with further signal 
processing steps which may be sensitive to any distortion of the signal introduced by the 


noise removal process. These are all topics for possible future study. 
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